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A  Microprocessor  local  computer  network  (MLCN)  test-bed  facility  was 
developed  at  the  Georgia  Institute  of  Technology  under  an  earlier  project 
[0'Re83].  One  of  the  features  of  this  MLCN  test-bed  is  the  facility  for 
emulating  local  area  networks  which  use  random  access  protocols.  This  projeot 
is  concerned  with  this  application  of  the  MLCN,  and,  more  specifically,  with 
verifying  the  ability  of  the  MLCN  to  emulate  local  networks  which  use  CSMA/CD 
(carrier  sense  multiple  access  with  collision  detection). 

The  ability  of  the  MLCN  facility  to  emulate  random  access  protocols 
accurately  has  been  assessed  using  simulation  data  and  data  from  all  available 
published  literature.  However,  there  exists  only  a  small  quantity  of  measured 
data  from  operating  networks.  Thus,  a  thorough  evolution  of  the  MLCN  facility 
with  respect  to  actual  operating  networks  has  not  been  possible. 

This  project  was  initiated  to  address  the  problem  of  evaluating  the  MLCN 
with  respect  to  an  operating  network.  The  project  had  two  general  goals 
namely:  1)  to  obtain  measured  data  and  develope  actual  operating  charac¬ 
teristics  from  an  in-service  random  access  type  local  network,  and  2)  to  use 
this  data  to  evaluate  the  ability  of  the  MLCN  facility  to  emulate  the  behavior 
of  such  a  network. 

The  choice  of  operating  network  for  the  project  was  limited  to  what  was 
available,  namely  an  Dngermann-Bass  Net/One  network.  Unfortunately,  the 
structure  and  characteristics  of  Net/One  are  very  poorly  documented  for  two 
reasons.  First,  Dngermann-Bass  regards  many  features  of  the  network  as  com¬ 
pany  confidential  and  aeoondly,  measurement  of  operating  characteristics  is  a 
non-routine  task  and  certainly  a  nontrivial  one. 

Achieving  the  first  of  the  project  goals  required  a  number  of  tasks 
whloh  Included:  design  of  measurement  software  and  hardware,  design  of 
experiments,  obtaining  appropriate  data,  and  constructing  models  for  the 


Page  -4- 


Nat/One  for  several  conditions  of  operation. 

Given  measured  characteristics  of  Net/One  and  a  model  for  its  operation 
under  varying  conditions,  the  second  part  of  the  project  is  to  emulate  the 
structure  of  Net/One  on  the  MLCN  facility  and  measure  essentially  the  same 
data  as  was  measured  on  the  operating  network.  A  comparison  of  these  measured 
results  would  serve  to  evaluate  the  accuraoy  of  the  emulator. 

It  was  clear  at  the  beginning  of  the  project  that  the  approach 
envisioned  entailed  certain  limitations.  The  most  Important  of  these  are: 
limitations  Imposed  because  of  available  equipment  (e.g.  the  Net/One  instal¬ 
lation  has  something  less  than  one  dozen  stations),  the  fact  that  measurement 
and  modeling  of  Net/One  had  to  take  place  before  emulation  could  be  carried 
out,  and  the  fact  that  the  complexity  of  the  Net/One  architecture  was  not 
known  at  the  Inception  of  the  project.  As  is  discussed  in  the  conclusions  to 
this  report,  all  of  these  factors  affected  the  breadth  of  the  study. 
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To  discuss  tbs  project  In  tbs  proper  perspective  it  is  desirable  to 
provide  some  background  on  typical  architectures  for  random  access  networks. 

Two  organizations  have  been  involved  in  developing  standards  for  networks, 
namely  the  International  Organization  for  Standardization  (ISO)  and  the 
Institute  of  Electrical  and  Electronic  Engineers  Project  802  Standards  Commit¬ 
tee  (IEEE  802). 

To  minimize  design  complexity,  modern  networks  tend  to  use  a  layered 
architecture  with  each  layer  being  a  logieal  entity  which  performs  certain 
functions.  The  servioes  performed  hy  the  highest  layer  are  for  the  network 
users.  Lower  layers  each  provide  services  for  the  next  higher  layer,  shield¬ 
ing  it  from  the  details  of  how  the  services  are  provided.  This  hierarchical 
structure  can  be  used  to  provide  all  of  the  intermediate  tasks  and  functions 
required  to  oarry  out  desired  user- to- user  interaction  over  the  network. 

When  users  located  at  different  network  nodes  communicate,  the 
corresponding  layers  at  each  node  also  communicate.  A  well-defined  set  of 
rules  called  a  "protocol"  is  necessary  for  each  level  to  carry  out  its  conver¬ 
sation  in  an  orderly  structured  manner.  These  protocols  establish  what  can  be 
regarded  as  logical  communication  paths  between  communicating  entities,  which 
are  called  "peer  processes". 

Protocols  are  organized  in  a  hierarchial  fashion  to  correspond  to  the 
network  layers.  All  but  the  lowest  layer  protocols  control  conversations 
across  a  single  layer  over  a  logical,  or  virtual,  communication  path.  The 
lowest  layer  protocol  controls  data  flow  over  the  physical  connecting  channel. 

The  number  of  layers,  the  name  of  each  layer,  and  the  function  of  each 
layer  differs  from  network  to  network.  The  two  extreme  layers  are  usually  the 
application,  (or  user-user),  and  physical  layers.  The  application  layer,  as 
its  name  implies,  deals  with  the  specific  application  of  the  user.  It  is 


always  the  highest  level.  The  physical  layer,  on  the  other  hand,  is  always 
the  bottom  layer,  as  it  deals  with  the  transmission  of  physical  signals 
between  the  two  nodes.  The  physioal  layer  includes  the  physical  trnamiaaion 
channel  and  provides  an  electrical  connection  between  the  communicating 
processes. 

Two  types  of  network  services  typically  available  are  termed  "datagram" 
and  "virtual  circuit"  in  the  ISO  literature.  Datagram  service  is  the  more 
basic  of  the  two  just  named.  In  providing  such  a  service,  the  network  and 
lower  layers  couple  the  sender  and  receiver  by  transferring  packets  in  a  com¬ 
pletely  Independent  manner  over  the  virtual  connection.  Datagram  service  can 
be  likened  to  the  postal  service  in  that  packets  are  delivered  to  a  destina¬ 
tion  address,  which  must  be  specified  in  the  packet,  but  such  delivery 
constitutes  all  that  is  required.  No  error  control  or  end-to-end  ack¬ 
nowledgements  are  provided,  and  datagrams  are  passed  to  the  user-host  in  the 
order  in  which  they  arrive. 

As  can  be  seen  from  the  above  description,  datagram  service  is  fairly 
primitive.  Virtual-circuit  servloe  is  at  the  other  extreme,  providing  all  of 
the  functions  noted  above  as  being  absent  from  the  datagram  service.  Thus  the 
virtual-circuit  service  not  only  delivers  each  packet  to  its  destination  but 
also  provides  on  an  end-to-end  basis  proper  sequencing  of  packets,  error 
control,  flow  control,  and  acknowledgements. 

Figure  1  shows  the  levels  identified  in  the  ISO  model  and  those 
specified  by  the  IEEE  802  committee.  Note  that  the  seven  layers  in  the  ISO 
model  can  be  divided  into  end-to-end  protocols  and  network  access  protocols. 
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In  reviewing  the  layered  structure  it  should  be  noted  that  the  IEEE  802 
standard  layers  -  physical,  media  access  control  (MAC)  and  logical  link 
control  (LLC)  -  encompass  esentially  all  that  is  needed  for  network  access. 
For  the  ISO  model,  the  group  of  layers  for  network  access  provide  all  of  the 
necessary  features  and  functions  required  to  provide  basic  datagram  or  virtual 
circuit  services,  although  actual  delivery  of  these  services  is  a  function  of 
the  transport  level  in  the  ISO  model.  Most  additional  services,  provided  by 
levels  higher  than  the  network  access  levels,  are  no  longer  in  the  area  of 
basic  communication  but  fall  into  application  specific  tasks  or  services  which 
are  not  of  interest  to  this  project. 


1.  PRELIMINARY  DESIGN  PE  EXPERIMENTS 

The  basic  operation  of  a  local  area  network  such  as  the  Net/One  can  be 
viewed  in  terms  of  the  datagram  or  virtual  circuit  services  that  it  provides. 
As  discussed  in  Section  2  of  the  report(  both  types  of  service  Involve  the 
transfer  of  packets  between  source  and  destination  nodes.  The  network  per* 
formance  in  transferring  packets  can,  in  turn,  be  characterized  by  relating 
throughput  and  average  delay  to  to  offered  traffic. 

Offered  traffic  is  defined  as  the  average  input  traffic  rate,  in  packets 
per  second,  to  each  node.  Total  offered  traffic  is  the  sum  of  the  average 
input  traffic  to  all  nodes  on  the  network.  Throughput  as  used  in  this  report, 
is  a  normalized  measure  of  average  output  traffic  either  from  some  specific 
node  or  from  the  complete  network.  It  is  defined  as  the  average  output 
traffic,  in  packets  per  second,  from  some  portion  of  the  network  divided  by 
the  maximum  packet  rate  of  the  output  channel.  The  packet  rate  of  the  output 
channel  is  the  bit  rate  or  clock  rate  of  the  ohannel  divided  by  the  average 
packet  length. 

As  a  final  definition,  the  average  delay  for  a  packet  is  the  delay  in 
seconds,  averaged  over  a  reasonable  length  of  time,  for  a  packet  to  traverse  a 
given  portion  of  the  network.  The  average  delay  normally  consists  of  waiting 
time  in  a  buffer,  propagation  delay  and  the  packet  transmission  time  required 
after  processing  of  the  packet  begins.  Average  delay  can  be  measured  over  any 
portion  of  the  network.  End-to-end  delay  and  round-trip  delay  are  two  types 
of  delay  often  cited. 

The  major  thrust  of  the  subject  project  can  be  stated  in  terms  of  the 
variables  defined  above  as  follows:  1)  For  Net/One  operating  under  a  set  of 
specified  conditions,  measure  the  throughput  and  average  delay  for  a  sequence 
of  offered  traffic  values,  2)  Use  this  and  other  data  to  establish  the  struc¬ 
ture  of  Net/One,  3)  Emulate  the  structure  of  Net/One  on  the  MLCN  facility,  and 
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■ake  Measurements  of  throughput  and  average  delay  which  correspond  to  this 
mode  for  Net/one,  4)  Compare  the  results.  One  of  the  tasks  of  the  project  was 
to  design  the  experiments  by  choosing  the  conditions  under  which  throughput 
and  delay  are  measured. 

The  general  structure  of  the  Net/One  is  shown  in  Figure  2.  The  end-to- 
end  average  packet  delay  between  two  user  devices  can  be  broken  down  into  a 
delay  for  each  of  the  components  of  the  path  identified  as  network  processor , 
transceiver  interface,  etc.  An  alternative  breakdown  of  delay  into  components 
would  use  the  layers  discussed  in  Section  2.  Classified  in  this  way,  using 
the  IEEE  802  notation  results  in  the  following  components:  delay  due  to  the 
physical  transmission  medium  (cable),  media  access  delay,  logical  link  control 
delay  and  delay  due  to  hlghr  level  protocols.  The  first  two  layer  delays  are 
asociated  with  the  cable,  the  transceiver  and  the  transceiver  interface  in 
Figure  2,  while  the  others  are  associated  with  the  network  processor. 

The  MLCN  facility  was  designed  to  emulate  directly  the  physical  trans¬ 
mission  medium  and  the  media  access  protocol. 

At  the  Inception  of  the  project  it  was  planned  to  duplicate  the 
significant  aspects  of  the  Net/One  network  processor,  the  operation  of  which 
is  primarily  software,  with  software  running  on  two  MLCN  computers  which 
normally  serve  as  the  physical  stations  for  the  MLCN  facility.  (See 
References  [1]  for  a  complete  description  of  the  MLCN  facility.)  The  approach 
would  result  in  an  emulation  for  statlon-to-station  communication  in  Net/One 
as  shown  in  Figure  3. 

A  consideration  of  the  overall  emulation  problem  or  the  related  problem 
of  characterizing  Net/One  led  to  a  two-step  approach.  In  a  first  step  the 
delays  and  characteristics  of  the  physical  and  media  access  layers  would  be 
measured  and  emulated.  This  would  then  be  followed  by  the  second  step  of 
working  with  oomplete  end-to-end  communication  between  two  users  ooupled  to 
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different  stations.  The  remainder  of  the  report  Is  structured!  to  a  large 
extent  around  these  two  steps  In  the  procedure. 
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Figure  3.  Emulation  of  Station-to  ‘Station  Communication  for  Nat/Ona. 
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In  order  to  fully  characterise  the  operation  of  the  LAN  network  under 
study.  It  was  necessary  to  accurately  determine  the  value  of  a  number  of 
parameters  describing  the  operation  and  performance  of  the  system.  The 
specific  parameter  values  required  were: 

e  Size  of  the  data  packets  -  the  size  of  the  packets  placed  on  the 

cable  and  the  number  of  device  data  characters  per  packet  (not  the 
total  size  of  the  ETHERNET  packet  data  field).  In  our  LAN  these  range 
from  1  to  64  data  characters,  with  flow  control  invoked  at  42. 

e  Delays/time  intervals 
#e  End-to-end  character  delivery  delays 
eee  Packet  building  delays 
eee  Protocol  delays 
•ee  Cable  delays 

mm  Access  delays 
eeee  Transit  delays 


It  was  necessary  to  obtain  values  for  these  parameters  under  various  loading 
conditions  with  respect  to  both  device  speeds  and  number  of  active  devices 
being  serviced  as  well  as  the  cable  loading.  All  of  these  parameters  are  ran¬ 
dom  variables  whose  underlying  distributions  were  not  known  and  could  not  be 
assumed.  This  made  it  necessary  to  capture  and  record  a  large  number  of 
observed  values  and  analyze  the  resulting  histograms  depicting  their 
distribution. 

1*1  General 

The  Ungermann-Bass  Net/One  is  a  10  Mbs  network  utilizing  the  Ethernet 
specification  at  the  physical  and  data  link  layers.  The  primary  user  inter¬ 
face  is  virtual  circuit  service  provided  by  Z60  processors  operating  in  a  mul¬ 
tiple  processor  configuration.  Each  node  of  the  network  is  housed  in  a  single 
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box  containing  up  to  four  processor  boards,  a  transmitter  board  and  a  receiver 
board  all  operating  as  one  unit  called  the  Network  Interface  Unit  (NIU). 


Eaob  NIU  consists  of  one  Network  Processor  Board  (NPB)  and  up  to  three 
Application  Processor  Boards  (APBs).  The  AFBs  are  all  optional,  but  all  the 
NIUs  used  in  these  experiments  had  three.  The  user  ports  are  on  the  four 
processor  boards  with  six  ports  on  a  board.  The  NIU  is  attached  to  a  radio- 
frequency  transceiver.  The  transceiver  is  connected  directly  to  the  ooaxlal 
cable  transmission  medium.  Figure  A  Illustrates  the  Net/One  NIU  organisation. 

The  primary  user  services  provided  are  interfaces  to  asynchronous, 
start-stop,  ohara oter-mode  DTE' a,  terminals,  computers,  or  other  perpherals. 
ETHERNET  [DIX  80]  is  utilised  to  provide  the  media  aocess  control  mechanism 
and  lower  level  protocols.  Slnoe  the  ETHERNET  system  is  fundamentally  a  pac¬ 
ket  interconnection  system,  the  individual  characters  transmitted  by  the 
attached  devloes  must  be  assembled  into  packets  before  transmission  (and 
disassembled  prior  to  delivery).  In  addition,  the  ETHERNET  specification 
provides  only  "unreliable  datagram  service,"  while  the  interconnection  of  two 
devices  require  a  reliable  connection  service.  Therefore,  it  is  necessary  for 
the  NIU' s  to  also  Include  a  Virtual  Clreult  Protocol. 

1*2.  The  Packet  Building  and  Buffer  Management  Processes 

The  attached  devloes  send  individual  characters  utilizing  the 
asynchronous,  start-stop  protoool.  As  each  character  is  received,  it  is 
processed  by  the  serial  interface  ohlp  and  placed  in  a  port-buffer  in  the 
Interface  Controller  memory.  Associated  with  each  device  Interface  port  is  a 
pool  of  buffers  utilized  to  assemble  characters  into  packets  for  transmission 
over  the  media.  The  paoket  building  and  buffer  management  processes  are  the 
most  important  activities  effecting  the  performance  of  the  LAN  system  under 
study.  Figure  5  illustrates  the  movement  of  data  and  the  transfer  of  buffers 
from  t ,n*>  activity  to  another.  Figure  6  Illustrates  the  timing  relationships 
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between  the  various  activities  that  take  place  and  gives  an  overall  view  of 
the  network  operation. 

As  in  almost  every  other  similar  system,  buffer  management  is  a  critical 
issue.  In  this  particular  system,  dedicated  buffers  are  assigned  to  each 
port.  The  operational  seheme  la  depicted  in  Figure  5.  There  are  three 
buffers  assigned  to  eaeh  port,  and  all  ports  function  independently  (up  to  a 
point)  and  in  parallel.  A  BUFFER-SERVICE- INTERRUPT  occurs  upon  the  expiration 
of  a  timer  in  the  Processor  Board  (16.4  milliseconds).  Vhen  this  Interrupt 
occurs,  the  processor  checks  to  determine  if  it  is  possible  to  close  off  the 
current  buffer  and  move  it  to  the  S END-QUEUE  for  transmission  to  the  other  end 
of  the  virtual  circuit.  A  port  buffer  is  closed  and  moved  to  the  S END-QUEUE 
if  either  of  the  following  conditions  apply: 

e  There  is  another  empty  buffer  immediately  available  to  continue  to 
receive  input  characters,  jan 

e  The  current  buffer  is  completely  filled. 

After  the  buffer  has  been  closed,  the  virtual  circuit  header  and  the 
ETHERNET  header  are  added  to  the  data  character(a).  It  nay  be  necessary  to 
add  up  to  eight  filler  characters  to  meet  ETHERNET  specifications  for  a 
minimum-size  packet. 

Figure  6  is  a  time- sequenced  chart  of  what  happens  to  an  individual  pac¬ 
ket  as  it  moves  through  the  LAN  from  one  NIU  to  another.  One  cannot  help  but 
notice  a  similarity  between  progression  of  data  through  the  NIU  and  movement 
of  data  through  a  classical  atore-and-forward  type  of  network.  It  should  not 
be  surprising  then  that  most  of  the  same  design  and  performance  Issues  are 
present.  These  are  flow  oontrol,  congestion,  addressing,  and  throughput 
problems.  Even  routing  Issues  are  present  as  the  virtual  ciroult  addressing 
is  processed. 

The  completed  ETHERNET  packet  la  tranaferred  to  the  S END-QUEUE  for  that 
Interface  Controller  by  moving  a  pointer,  as  depleted  in  Figures  5  4  6.  The 
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SEND-QUEUE  Is  a  FIFO  (first-in,  first-out)  service  order  queue.  There  are 
separate  S END-QUEUEs  for  each  port  controller  board  (i.e.  APB  or  NPB)  in  the 
systam.  There  can  be  up  to  four  boards  in  this  product.  Each  controller 
board  may  have  up  to  alz  porta.  The  node  controller  continually  polls  all  of 
the  Processor  Boards  to  determine  if  there  is  a  packet  ready  to  be  transmit¬ 
ted.  There  is  a  classical  random  queuing  delay  at  this  point  that  has  several 
distinct  parts.  The  buffer  must  wait  until  it  reaches  the  head  of  the  queue. 
This  delay  is  a  function  of  the  number  of  buffers  ahead  of  it,  the  service  or 
transfer  time  for  a  buffer,  and  the  wait  for  the  next  poll  by  the  Node 
Controller  to  service  the  SEND-QUEOE  of  a  particular  controller  board.  From 
one  of  the  SBJD-QUEUEs,  on  a  port  board,  the  packet  is  copied  to  the  TRANSMIT- 
BUFFER  on  the  node  controller  board. 

In  this  system  the  copy  is  by  DMA  transfer.  The  buffer  cannot  be 
released  from  the  SEND-QUEUE  for  reuse  at  this  time,  due  to  the  guaranteed 
delivery  or  Virtual  Circuit  requirement.  It  must  be  held  here  until  either  an 
acknowledgement  is  received  or  a  timeout  occurs  triggering  a  retransmission  of 
this  packet. 

The  packet  in  the  TRANSMIT-BUFFER  now  experiences  another  random  delay. 
When  the  packet  reaches  the  head  of  the  TRANSMIT-BUFFER  queue,  the  random  ser¬ 
vice  delay  has  several  components  of  significance.  Here  is  where  the  access 
oontrol  mechanism  comes  into  view.  If  it  is  contention  based  (ETHERNET,  as 
our  case)  the  cable  acoess,  collision  resolution  (random,  truncated  binary- 
exponential  delay),  and  retransmission  delays  must  be  counted.  The  final  com¬ 
ponent  of  this  time  delay  is  the  transmit  time  which  is  a  linear  delay  based 
on  10  MBS  cable  speed  and  ETHERNET  packet  specification.  The  range  is  from 
67.2  microsec  minimum  packet  size  up  to  1.23  milliseconds  for  a  maximum  1500 
octet  packet. 

After  the  ETHERNET  packet  has  been  successfully  received,  the  oomplete 
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packet  is  placed  In  the  RECEIV E-BUFFER  In  the  receiver.  The  ETHERNET  header 
oust  be  retained  for  the  higher  level  acknowledgement,  so  the  aize  ia  not 
reduced.  The  node  controller  proceasea  the  RECEIVE-QUEUE  in  FIFO  order. 
Another  classical  queueing  delay  is  experienced.  The  virtual  circuit  header 
is  checked  to  determine  where  to  move  the  buffer.  This  processing  time  la 
still  random,  but  well  behaved  as  compared  with  some  of  the  others.  It  ia  a 
function  of  the  total  activity  of  the  Cable  Interface  Unit  and  the  competition 
for  CPU  cycles.  Once  the  location  ia  determined,  the  packet  is  copied  into  a 
RECEIVE-QUEUE  on  a  port  controller  board.  Net/One  also  uses  DMA  for  this 
transfer.  Again  random  queueing  and  service  delays  are  experienced.  There  is 
a  wait  for  the  DMA  to  be  aet-up,  and  then  the  copy  time  is  a  function  of  the 
packet  length  and  the  CPU-cycle  competition. 

In  the  RECEIVE-QUEUE,  the  virtual  circuit  header  is  examined  to 
determine  which  output  port  is  to  receive  the  data,  and  where  to  return  the 
acknowledgement.  An  ACK-packet  ia  constructed  and  set  up  for  transmission 
back  to  the  sending  controller  board.  Meanwhile,  the  buffer  is  moved  by  a 
pointer  change  from  the  RECEIVE-QUEUE  to  e  particular  port  controller.  It  is 
serially  transmitted  to  the  destination  device  at  a  data  rate  determined  by 
that  device  and  subject  to  its  flow  control.  The  delays  are  proportional  to 
the  bit  rate  and  the  length  of  the  raw  data  with  all  headers  removed.  The 
acknowledgement  packet  traverses  the  system  in  the  same  manner  just  described, 
back  to  the  originating  port  controller  board.  Figure  6  shows  the  additional 
time  for  processing  the  return  ack  so  that  the  buffer  can  be  released  for 
reuse. 
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JU1  Data  structures 


A  large  number  of  data  structures  are  required  to  handle  all  2k  virtual 
circuits  on  the  four  processor  boards  in  an  NIU.  Each  processor  board  has  one 
transmit  control  block  (XCB)  and  an  array  of  thirteen  data  type  descriptor 
control  blocks  (DTDs).  Each  XCB  consists  of  a  control  word  used  to  determine 
if  the  XCB  is  in  use  and  valid,  an  address  field  used  as  the  source  of  the 
current  packet  and  an  address  for  the  start  of  the  next  packet.  A  pointer 
points  to  one  of  the  buffers  on  the  send  queue,  which  is  a  linked  list  of  pac¬ 
kets  ready  to  be  sent.  The  DTDs  contain  information  similar  to  that  in  the 
XCB  in  addition  to  the  linked  list  of  fill  and  receive  buffers  into  which  the 
NPB  can  perform  direct  memory  access  (DMA)  of  packets  addressed  to  the 
associated  DTD.  Fill  buffers  are  used  by  the  NPB  as  the  destination  of  DMA 
transfers  from  the  receiver  board.  Receive  buffers  are  seen  by  the  receiving 
process  as  the  next  to  empty.  All  data  structures  are  stored  in  shared  memory 
to  permit  the  NPB  to  access  them.  A  process,  in  order  to  be  able  to  have 
access  to  the  transmission  medium,  must,  on  Initialization,  request  a  data 
type  from  the  network  manager.  It  can  request  a  specific  data  type  for  a 
special  purpose  or  can  request  for  the  next  available  data  type.  Thus,  data 
types  are  associated  with  processses  and  are  addressed  by  the  processes  to 
which  they  are  assigned. 

2.2  JSLXQL  Packet  Transmission  Process 

All  data  transfer  out  of  the  NIU  is  handled  by  the  transmitter  process 
on  the  transmitter  board.  The  transmitter  process  consists  of  a  kK  byte  FIFO 
buffer,  direct  memory  access  (DMA)  logic  to  enable  the  transmitter  to  obtain 
data  directly  from  another  processor's  memory,  and  the  logic  to  detect  col¬ 
lisions.  Collisions  are  detected  by  the  transmitter  monitoring  all  data 
transmissions  and  doing  a  bit  by  bit  comparison  between  the  transmitted  and 
received  data.  The  Interface  between  the  NPB  and  the  transmitter  process  is 
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provided  by  the  Z80  I/O  ports  which  addresses  command  registers  on  the  trans¬ 
mitter  board.  The  NFB  scans  the  XCBs  of  the  processor  boards.  If  it  finds  an 
XCB  which  is  ready  and  valid,  it  sets  up  a  DMA  transfer  directly  to  the  FIFO 
of  the  transmitter  board  by  transfering  to  the  transmitter  register  the  source 
address,  the  byte  count  and  the  DMA  enable  command.  DMA  then  begins,  and  is 
controlled  by  the  transmitter  process  until  the  byte  count  reaches  zero.  At 
the  zero  byte  count,  the  transmitter  issues  an  interrupt  to  the  interface 
process  running  on  the  NPB.  The  NPB  then  enables  the  transmitter  to  transmit 
the  packet  as  the  transmission  medium  permits.  As  soon  as  the  packet  has  been 
successfully  transmitted,  the  transmitter  process  notifies  the  NPB.  The  XCB 
of  the  transmitted  packet  continues  to  be  marked  as  in  use  until  an  ack¬ 
nowledgement  to  the  packet  has  been  received.  Figure  7  illustrates  the  NIU 
transmission  process. 
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A.5.  NIP  Packet  Receiving  Process 

Receiving  a  packet  is  done  by  the  receiver  process  which  runs  on  the 
receiver  board.  The  board  contains  a  4K  bytes  of  FIFO  buffer,  DMA  logic,  com¬ 
mand  registers,  and  a  32-bit  cyclic  redundancy  check  (CFC)  logic.  The 
receiver  automatically  checks  the  Ethernet  header  on  all  packets  to  determine 
if  the  packet  is  addressed  to  its  node.  If  it  is  not,  the  receiver  ignores 
the  packet  and  continues  to  sense  the  carrier.  Otherwise,  the  receiver  begins 
at  the  current  location  in  its  FIFO  and  loads  the  data  into  it.  When  the  end 
of  the  packet  is  reached,  the  FIFO  pointer  is  updated  to  reflect  the  current 
location.  The  receiver  next  interrupts  the  NPB,  which  determines  the  proces- 
sor  board  to  which  the  packet  is  addresssed.  If  the  packet  is  good,  the  NPB 
checks  the  DTD  of  the  destination  processor  board  to  determine  if  a  buffer  is 
available.  If  not,  the  packet  is  discarded.  Otherwise,  a  DMA  transfer  from 
the  receiver  FIFO  is  set  up  to  the  fill  buffer  and  a  control  word  is  set  to 
indicate  that  tha  buffer  is  in  use.  The  NPB  then  interrupts  the  processor 
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board,  which  polls  its  DTDs  to  deteraine  which  one  has  a  ready  packet.  Any 
ready  packet  is  removed  from  the  <ill  queue  and  placed  on  the  receive  queue. 
The  APB  or  NPB  process  has  the  responsibility  to  notify  the  receiving  process 
of  a  packet  arrival  and  returning  the  empty  buffer  back  on  the  fill  queue. 
However,  it  is  the  responsibility  of  the  receiving  process  to  process  the  data 
in  the  packet  and  flag  the  receive  buffer  as  free.  Figure  8  illustrates  the 
NIU  receiving  process. 
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Figure  4.  Net/One  Hardware  Architecture. 
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£.  MEASPREMENT  TOOLS  AWD  TECHNIQUES 


The  experiments  were  performed  In  the  Computer  Laboratory  of  the  School 
of  Information  and  Computer  Science  at  the  Georgia  Institute  of  Technology 
using  the  Net/One  facility  described  above. 

Initial  experiments  were  designed  to  measure  the  performance  of  the 
physical  and  media  access  layers  in  the  Net/One  architecture.  These  layers 
satisfy  what  is  often  tensed  the  "Ethernet"  specification. 

To  carry  out  this  part  of  the  study  all  elements  of  Net/One,  as  shown  in 
Figure  2,  above  the  transmit  and  receive  controller  were  deactivated.  Traffic 
was  generated  by  specially  designed  software  and  injected  into  the  transmit 
controller. 

Experiments  consisted  of  injecting  traffic,  with  either  a  constant  or 
exponentially  distributed  time  between  arrivals,  into  the  transmit  controller 
on  each  of  the  five  stations.  A  monitor  station  was  then  used  to  measure 
network  throughput  as  a  function  of  input  traffic  load.  The  "load"  was 
determined  by  adjusting  the  average  time  between  packet  arrivals. 

Experiments  subsequent  to  the  initial  one  were  performed  with  Net/One  in 
its  normal  configuration.  Tools  and  techniques  for  these  experiments  are  now 
described. 

A  major  concern  in  our  study  of  the  Net/One  performance  was  understand* 
lng  how  the  NIUs  built  Ethernet  packets  from  the  input  data  and  determining 
the  size  distribution  of  those  packets  under  various  user  input  and  cable 
loads.  A  significant  group  of  experiments  examined  the  process  of  packet 
building  in  the  NIU  port  buffer.  Since  backoffs  from  a  busy  cable  could  have 
a  'back- pressure'  effect,  an  empty  cable  was  used  initially.  This  Isolated 
the  operation  of  port  buffer  management  so  that  its  behavior  was  affected  only 
by  the  ability  of  the  board  processor  to  handle  Incoming  trafflo. 
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5*1  Traffic  Generator 

Traffic  was  generated  with  a  character  generator  adjustable  to  traffic 
rates  of  109 ,  291  and  436  characters  per  second.  The  traffic  stream  could  be 
directed  into  any  number  of  ports  simultaneously.  Since  each  board  has  its 
own  dedicated  processor,  the  use  of  the  same  data  stream  for  simulating  user 
input  traffic  should  not  affect  the  timing  for  the  buffers.  Measurement  of 
packet  size  were  made  at  another  NIU  (the  monitor),  after  the  lower  level 
protocol  wrapper  (i.e. ,  Ethernet  and  Net/One  virtual  circuit  header),  had  been 
stripped  off.  As  each  packet  arrived,  the  size  field  in  the  Net/One  header 
was  examined  and  a  counter  in  an  array  indexed  by  the  packet  size  was 
Incremented.  Each  experiment  lasted  five  minutes,  long  enough  for  the  network 
to  stabilize.  Packet  sizes  of  37  bytes  were  ignored;  they  were  ack¬ 
nowledgement  frames.  All  packets  shorter  than  37  bytes  were  packet  fragments 
caused  by  collisions  and  were  also  Ignored. 

5.2  Packet  Length  Measurements 

Although  it  might  have  been  possible  to  obtain  data  on  the  length  of  the 
packets  generated  by  an  active  Cable  Interface  Unit  through  direct  measurement 
of  its  operations,  the  taking  of  such  data  would  probably  have  distorted  the 
operation  of  the  Controller  and  made  the  figures  obtained  inaccurate.  The 
method  chosen  was  to  utilize  a  separate  Network  Interface  Unit  operating  in 
the  permissive  mode,  i.e.,  receive  all  packets.  This  separate  NIU  permitted 
the  examination  of  the  virtual  circuit  header  and  recording  of  the  values  of 
the  length  fields  in  those  headers.  This  data  was  collected  and  presented  in 
the  form  of  histograms. 

5.3  Charaotsr  Delivery  Delay  Measurements 

A  special  measuring  device  was  constructed  to  measure  the  character 
delivery  delays.  This  measurement  device  has  the  capability  to  send  oharac- 
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ters  to  the  Network  Interface  Unit  at  selectable  constant  rates.  At  the  same 
tine  it  can  determine  the  delivery  delay  from  NIU  to  NIU  within  a  half  mil¬ 
lisecond  using  a  timestamp  technique.  The  device  collects  the  data  and 
presents  a  histogram  depicting  the  distribution  of  the  delivery  delays. 
Utilizing  this  measurement  device,  it  was  possible  to  run  a  wide  variety  of 
i  experiments  for  a  length  of  time  sufficient  to  obtain  high  statistical  con¬ 

fidence  in  the  distributions  obtained. 

I 
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&.  MEASDBED  RESULTS 


4*1  iMMltl  XfiE  •Ethernet  Jlfift*  Deration 

Measured  data  for  the  initial  experiment,  involving  only  the  physical 
and  media  aooess  layers  of  Net/One,  is  plotted  in  Figure  9  as  throughput  ver¬ 
sus  packet  arrival  rate  at  each  station.  Five  stations  were  used,  each  with 
the  same  arrival  note.  Operating  in  the  somewhat  artificial  mode,  as 
described  in  the  last  section,  it  was  possible  to  load  the  network  to  a 
throughput  of  0.6  as  shown  in  Figure  9. 

&.J.  Packet  Length 

Figure  10  shows  the  distribution  of  packet  sizes  with  one  active  port  on 
the  APB  and  a  character  arrival  rate  of  109  character  per  second.  This 
arrival  rate  corresponds  to  about  1.78  characters  per  time  slice  of  16.4  mil¬ 
liseconds,  specified  in  the  Net/One  description  as  the  length  of  the  port 
buffer  cut-off  timer.  In  Figure  10,  it  can  be  seen  that  the  mean  of  the 
observed  packet  sizes  is  about  2.54  data  bytes.  This  indicates  that  buffers 
were  closed  at  approximately  18.4  or  27*6  milliseconds  time  interval.  This 
suggests  that  there  are  some  secondary  delays  after  the  interrupts  before  the 
processor  can  close  the  buffer  and  move  the  packet  to  the  send  queue. 

Figure  11  shows  an  exponential  distribution  of  packet  sizes.  There  were 
four  active  ports  on  an  APB, each  with  a  character  arrival  rate  of  109  per 
second.  As  in  Figure  10,  the  character  arrival  rate  per  time  Interval  of  16.4 
milliseconds  per  port  was  1.78.  This  suggest  that  as  the  number  of  active 
ports  Increases,  the  service  distribution  of  the  processor  becomes  more  ran¬ 
dom.  About  90?  of  the  packets  are  between  2  and  22  data  bytes.  With  a  mean 
packet  size  of  about  6  data  bytes,  buffers  are  servloed  on  an  average  of  about 
64  milliseconds  time  Interval  This  time  of  about  64  milliseconds  is  about  four 
times  the  buffer  time  interval  (16.4  ms),  suggesting  that  the  processor  is 


becoming  heavily  loaded  servicing  the  port  buffer. 

Figure  12  is  almost  identical  to  Figure  10.  Both  figures  have  similar 
characteristics.  The  only  difference  between  the  two  is  that  Figure  10  used 
an  APB  while  Figure  12  used  an  NPB.  The  similarities  in  both  figures  suggest 
that  at  low  character  rates,  there  is  no  difference  in  the  packet  size 
distribution  for  an  APB  or  an  NPB. 

In  Figure  13  the  packet  size  distribution  follows  a  normal  distribution 
with  a  mean  of  about  5  data  bytes.  This  distribution  is  in  contrast  to  Figure 
11.  As  may  be  recalled,  the  only  difference  between  the  measurement  charac¬ 
teristics  of  Figures  11  and  13  is  the  type  of  processor  board  used.  The  pac¬ 
ket  distribution  pattern  of  Figure  13  suggests  that  as  the  number  of  ports 
Increases,  the  buffer  servicing  rate  on  the  NPB  becomes  faster  than  that  on 
the  APB.  Using  a  mean  packet  size  of  5  data  bytes,  the  average  buffer  service 
Interval  is  about  45.9  milliseconds. 

Figure  14,  which  depicts  the  packet  size  distribution  with  one  port 
active  at  291  characters  per  second  (cps)  on  an  APB,  shows  packets  distributed 
equally  between  sizes  of  8  and  9  data  bytes.  With  a  mesa  packet  size  of  8.5 
data  bytes,  the  buffer  timer  is  about  29*2  milliseconds,  again  almost  twice 
the  length  given  in  the  Net/One  specification.  This  distribution,  that  is, 
all  packets  centered  about  two  values  indicates  availability  of  a  free  port 
buffer  at  all  times.  There  was  an  avilable  buffer  because  there  was  only  one 
port  active  in  the  entire  NXU  and  the  data  rate  was  moderate.  As  the  number 
of  active  ports  is  increased  to  4  as  shown  in  Figure  15,  buffers  were  open  for 
longer  time,  resisting  in  larger  packet  sizes.  This  suggests  that  as  the  num¬ 
ber  of  ports  increases,  the  servicing  time  for  the  port  buffers  becomes 
longer. 

The  mean  packet  size  in  Figure  16  is  9  data  bytes.  This  is  almost 
Identical  to  Figure  14.  The  average  buffer  length  of  the  service  time 
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corresponding  to  this  distribution  is  about  31  Billiseconds.  Again,  this 
repeats  that  buffer  timing  is  almost  double  the  number  given  in  the  Net/One 
specification.  Figure  17  she  the  same  behavior  as  Figure  18  and  can 
therefore  be  explained  similarlly. 

Figure  18  shows  one  active  port  on  NPB  at  a  character  arrival  rate  of 
436  per  second.  This  arrival  rate  corresponds  to  7*15  characters  per  the  16.4 
Billiseconds  time  Interval.  From  the  figure,  packets  are  distributed  around 
Bultlples  of  14,  that  is  14,  28,  42,  and  56  data  bytes.  This  packet  distribu¬ 
tion  means  that  buffers  were  closing  at  time  intervals  of  approximately  32 
Billiseconds.  This  packet  size  distribution  indicates  that  the  length  of  a 
packet  greatly  Influences  its  delay  in  the  port  buffer.  Thus,  high  character 
arrival  rates  results  in  longer  packet,  which  in  turn  Increases  buffer  servic¬ 
ing  timing. 

When  the  different  character  rates  are  compared,  there  is  a  trend 
towards  a  higher  minimum  packet  size.  As  noted  before,  at  109  cps  there  are 
almost  no  single  byte  data  packets,  but  about  90?  are  of  2  or  3  data  bytes. 
At  291  cps  the  minimum  data  packet  is  8  or  9  bytes,  and  at  436  cps  (Figure  19) 
the  smallest  peak  is  at  approximately  17  data  bytes.  The  size  of  the  data 
bytes  seems  to  depend  on  the  number  of  characters  arriving  during  a  time 
Interval,  but  it  is  almost  twice  the  expected  amount  based  on  the  16.4  mil¬ 
liseconds  time  value. 

Table  1 .  Smallest  significant  data  size  observed  and  data 
sizes  that  will  be  produced  assuming  16.4  ms 
port  buffer  service  Interval  with  one  active  port. 


CPS  Data  size  Smallest  significant 

per  16.4  ms  packet  size  observed 


The  second  group  of  experiments  (Figures  20  and  21)  examined  the  packet 
size  distribution  for  a  typical  traffic  load  on  the  network.  The  group  at  one 
data  byte  probably  results  from  low  speed  human  input.  The  peak  at  15  data 
bytes  indicates  a  process  that  produced  constant  character  rate  of  468  assum¬ 
ing  32  milliseconds  timer  for  port  buffers.  The  packet  sizes  of  70  data  bytes 
were  probably  produced  by  a  Xerox  Star  system  which  was  also  operating  on  the 
same  cable  during  this  experiment. 

The  third  group  of  experiments  involved  traffic  generated  by  a  typist 
with  a  background  load  simulated  by  a  special  software  packet  generator.  The 
expected  result  is  that  as  the  load  on  the  cable  increases  the  send  queue 
becomes  longer  resulting  in  long  delay  in  servicing  port  buffers.  This  will 
therefore  increase  packet  sizes.  But  the  results,  as  shown  in  Figures  22,  23 
and  24  do  not  Indicate  any  significant  change  in  packet  sizes  as  the  load  on 
the  cable  increased.  In  view  of  this,  the  conclusion  was  that  the  cable  may 
be  so  much  faster  than  the  NIU  that  its  load  is  not  a  factor  in  the  NIU 
operation. 

BttllY en.  Delay  Measurement? 

As  should  be  expected,  delivery  delays  showed  the  same  multi-mode 
distributions  as  packet  lengths.  Figure  24  is  the  distribution  of  delays  for 
one  port  active  at  1200  bps.  Most  experiments  have  shown  a  single  peak  around 
81  ms,  whereas  this  particular  one  has  only  34$  there,  with  52$  clustered 
around  61  ms.  The  rest  of  the  data  is  scattered  with  the  largest  being  3*6$ 
at  66  ms.  We  are  currently  trying  to  analyze  the  behavior  of  these  delay 
distributions. 

Figure  25  shows  the  distribution  of  delays  for  one  port  at  both  1200  and 
4800  bps.  For  the  4800  bps  experiment,  there  is  59$  clustered  at  79.5  ms,  40$ 
at  77  ms,  and  only  1$  scattered.  These  two  peaks  are  separated  by 
approximately  one  character  time  and  represent  the  one  character  uncertainty. 
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For  these  experiments  we  collect  data  for  about  300  seconds  for  each  run. 
Ahtough  the  packet  length  and  delay  time  measurements  are  not  synchronized, 
they  are  done  approximately  at  the  same  time  and  results  are  reasonably 
repeatable.  Not  every  packet  is  measured  at  the  faster  bit  rates,  but  a  ran¬ 
dom  sampling  is  done.  For  both  experiments  the  number  of  delay  measurements 
is  on  the  order  of  36000  packets. 
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Fiflura  10.  Packet  Size  Distribution  for  One  Port  on  APB  Active  at 
Character  Rata  of  109  Par  Second. 
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Figure  11.  Packet  Size  Distribution  for  Four  Ports  on  APB, 
All  Active  at  Rata  of  109  Characters  par  Second 
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Figure  12 
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Fi^ire  12.  Packet  Size  Distribution  for  One  Port  on  NPB  Active  at 
Rate  of  109  Characters  per  Second. 
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Figure  13.  Packet  Size  Distribution  for  Four  Ports  on  NPB. 

All  Active  et  Rate  of  109  Characters  per  Second. 
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Fipirt  14.  Packet  Size  Distribution  for  One  Port  on  APB, 
Activ*  at  Rata  of  291  Characters  par  Second. 
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Figure  15.  Packet  Size  Distribution  for  Four  Ports  on  APB, 
All  Active  at  Rate  of  291  Characters  per  Second 
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Figure  16.  Packet  Size  Distribution  for  One  Port  on  NPB, 
Active  at  Rate  of  291  Characters  per  Second. 
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Figure  17.  Packet  Size  Distribution  for  Four  Ports  on  NPB, 
All  Active  at  Rata  of  291  Characters  par  second. 
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Figure  10.  Packet  Size  Distribution  for  Four  Ports  on  NPB, 
All  Active  at  Rate  of  436  Characters  per  Second. 
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Figure  20.  Packet  Size  Distribution  for  •  Typical  Real  Working  Load 
Measured  Over  a  Period  of  30  Minutes:  Measurement  1. 
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Figure  21.  Packet  Site  Distribution  for  a  Typical  Real  Working  Load 
Measured  Over  a  Peroid  of  30  Minutes:  Measurement  2. 
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Figure  22.  Packet  Size  Distribution  for 
Background  Load. 
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Figure  23.  Packet  Size  Distribution  for  a  Typical  Typist  with  a  40%  Load  of 
128  Byte  Packets  also  Applied. 
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Figure  24.  Packet  Size  Distribution  for  a  Typical  Typist  with  a  50%  Load  of  12 
128  Byte  Packets  also  Applied- 
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Figure  25.  Character  Delivery  Deley  Distribution*. 
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1.  EMULATION  OF  KET/OHE 


As  noted  above,  the  eoulation  of  Net/One  ia  undertaken  in  two  parts, 
first  the  media  access  and  physical  layers  are  enulated  and  then  the  higher 
layers  which  provide  for  virtual  circuit  operation. 

2.1  Basic  Media  Access  Operation 

As  discussed  in  [Murr83]  and[0'RE83],  the  MLCN  facility  is  structured  to 
emulate  the  physical  and  media  access  layers  of  a  CSMA/CD  type  local  area 
network  in  a  fairly  direct  fashion.  Provision  is  made  for  two  primary 
stations  and  for  a  specified  number  of  background  stations.  Time  delays, 
corresponding  to  propagation  delays  over  varying  lengths  of  cable,  can  be 
programmed  between  the  two  primary  stations,  and  between  each  of  the  primary 
stations  and  the  effective  location  of  the  background.  Average  arrival  rate, 
average  packet  length  and  number  of  stations  are  parameters  of  the  background. 
The  arrival  times  of  packets  for  the  background  are  structured  to  be  those  of 
a  Poisson  process.  Traffic  to  the  primary  stations  must  be  supplied  exter¬ 
nally. 

As  discussed  in  Sections  5  and  6,  the  basic  media  access  experiment  was 
carried  out  on  the  Net/One  with  a  toted  of  five  stations  each  operating  to 
supply  1/5  of  the  total  load.  Throughput  versus  load  was  measured  for  the 
whole  network  with  constant  length  packets.  The  case  for  which  a  direct 
emulation  can  be  made  used  an  exponentially  distributed  time  between  packet 
arrivals.  The  total  cable  length  used  by  the  Net/One  was  very  short, 
corresponding  to  0.212  microsec  of  delay. 

In  preparing  for  the  emulation,  the  average  packet  length  for  constant 
length  packets  and  the  average  time  between  packets  are  specified  dlreotly 
from  the  Net/One  data. 

The  decision  was  made  to  use  one  primary  station  and  four  stations  in 
the  background  for  the  emulator.  A  program  was  written  for  the  Nova  computer 
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to  provide  constant  length  packets  with  an  exponentially  distributed  time 
between  arrivals  to  the  single  primary  station.  The  average  arrival  rate  was 
varied  through  the  same  sequence  of  values  used  with  Net/One. 

Either  of  the  two  configurations  shown  in  Figure  26  could  be  used. 
However,  the  very  small  propagation  delay  (due  to  the  short  cable)  makes  it 
immaterial  which  is  used. 

A  study  of  the  Ethernet  specifications  for  the  physical  layer  indictes 
that  there  can  be  several  equipment  delays  of  the  same  order  of  magnitude  as 
the  cable  delay  for  a  very  short  cable.  Since  instrumentation  is  not 
available  for  measuring  these  delays,  a  discrete-event  simulation  study  was 
carried  out  comparing  throughput  versus  arrival  rate  curves  for  different 
values  of  total  delay  to  those  obtained  from  Net/One.  A  total  delay  of  2  msec 
gives  the  best  results  and  this  value  of  delay  was  used  for  the  emulator. 

Throughput  vs.  offered  traffic  (arrival  rate)  curves  are  given  in 
Figure  27  for  the  emulator  in  comparison  to  the  curve  for  Net/One.  A  good 
agreement  up  to  large  values  of  load  can  be  noted. 

7*2  Virtual  Circuit  Operation 

Emulation  of  all  layers  of  the  Net/One  protocol  necessary  for  virtual 
circuit  operation  was  dlcussed  briefly  in  Section  3  and  a  preliminary  con¬ 
figuration  for  the  emulator  is  given  in  Figure  3«  This  approach,  planned  in 
the  early  part  of  the  project,  envisioned  proramming  the  essential  parts  of 
the  higher  level  Net/One  protocols  on  the  Nova  computers  used  at  the  primary 
nodes  of  the  emulator. 

Unfortunately,  subsequent  data  from  the  Net/One  showed  that  this 
approach  cannot  be  used.  As  is  discussed  in  Section  A,  the  Net/One  does  not 
use  straight  forward  higher  layer  protocols  programmed  in  software.  Instead, 
as  discussed  in  Section  A,  the  packetlzlng  process  is  carried  out  with  a  very 
complicated  combination  of  hardware  and  software  which  would  be  difficult  if 
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not  in possible  to  represent  with  software. 

The  above  facts  becane  clear  late  In  the  project  and,  although  several 
alternatives  to  the  structure  for  the  emulator  shown  in  Figure  3  were 
considered,  no  feasible  general  approach  was  discovered. 

An  alternative,  and  admittedly  special  purpose,  approach  to  emulating  at 
least  a  part  of  the  Net/One  behavior  was  chosen.  Characteristics  of  Net/One 
were  measured,  as  discussed  in  Sections  5  and  6,  under  conditions  such  that 
the  delay  caused  by  the  higher  level  protocols  is  essentially  constant.  Under 
these  conditions,  variation  of  average  access  delay  with  load  is  primarily  due 
to  the  delays  encountered  in  the  physical  and  media  access  layers.  Under  con¬ 
ditions  of  light  load,  these  delays  are  negligible  in  comparison  to  those  of 
the  higher  level  operations.  Unfortunately,  heavy  loads  cannot  be  achieved 
with  the  number  of  stations  we  had  available  on  the  Net/One. 

TWo  sets  of  emulation  data  were  taken.  The  first  emulates  the  packet 
length  and  arrival  rates  for  a  typical  experiment  on  Net/One  and  adds  back¬ 
ground  stations  with  essentially  the  same  packet  length  and  arrival 
statistics.  Figure  28  shows  the  throughput  of  one  primary  station  versus  the 
number  of  background  stations  under  the  conditions.  Note  that  adding  up  to 
275  stations  does  not  affect  throughput. 

A  second  experiment  was  designed  to  increase  the  background  load  until  a 
notlcable  decrease  in  the  throughput  of  the  primary  station  occured.  To 
accomplish  this  the  average  arrival  rate  to  the  background  stations  was 
Increased  to  180  packets/second  and  the  number  of  stations  was  varied  from  20 
to  275.  The  results  are  shown  in  Figure  29. 
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Figure  28.  Throughput  of  One  Primary  Station  Venus  Number  of  Background 
Stations  for  Typical  Net/One  Loading  Conditions .  Arrival  Rate 
Is  60  Packets  Per  Second. 
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Figure  29.  Throughput  of  Station  A  Versus  Number  of  Background  Stations. 


1.  Conclusions 

The  results  of  the  project  have  been  useful,  and  possibly  significant, 
but  not  exactly  in  the  directions  anticipated.  Determining  the  charac¬ 
teristics  of  Net/One  turned  out  to  be  a  much  more  difficult  and  time  consuming 
task  than  anticipated.  The  measured  characteristics,  when  obtained,  showed  an 
architecture  that  is  not  amenable  to  emulation  on  the  MLCN  as  presently  struc¬ 
tured.  The  most  significant  accomplishment  of  the  project  thus  turned  out  to 
be  the  characterization  of  Net/One.  The  majority  of  this  section  is  devoted 
to  the  characterization  of  Net/One  after  a  brief  summary  of  the  emulation 
results. 

Conclusions  with  flesoect  to  Emulation  of  Net/One 

The  MLCN  emulator  is  tailored  to  emulate  the  Ethernet  levels  of  a 
network.  Therefore,  not  surprisingly,  the  results  from  the  emulator  compared 
favorably  with  the  results  obtained  by  measurement  of  the  physical  and  media 
access  layers  of  Net/One  as  shown  in  Figure  26. 

Since  it  is  virtually  impossible  to  emulate  the  higher  layers  of  Net/One 
on  the  MLCN  as  presently  structured,  the  study  was  limited  to  determining  the 
effect  of  background  loading  on  the  operation  of  one  Net/One  station.  The 
results  presented  in  Figure  28  show  that  under  typical  conditions  the  physical 
and  media  access  layers  have  negligible  effect  on  average  packet  delay. 
Figure  29  shows,  however,  that  it  is  possible  to  load  the  cable  heavily  enough 
so  that  the  physical  and  media  access  layers  affect  delay.  Under  the  con¬ 
ditions  used  to  otain  the  data  for  Figure  28,  approximately  250  stations  are 
required  to  affect  the  throughput  of  the  primary  station  significantly. 

£.2  OfrAWYatlQBa  and  Conclusions  From  Measured  Data  on  Net /One 

Measurements  show  that  with  a  single  port  operating  at  1200  bits  per 
second  and  no  other  activity  on  the  network,  the  delivery  delay  for  one 


character  la  approximately  80  milliseconds.  Of  this  total  delay,  8.3  mil- 
liseconds  can  be  attributed  to  the  serial  transfer  from  the  device  to  the  port 
buffer  and  another  8.3  milliseconds  is  required  for  the  output  transfer,  and 
on  the  average,  there  is  an  uncertainty  of  one-half  character  transfer  time, 
or  another  4  ms.  The  other  59  milliseconds  are  delivery  processing  and  trans¬ 
fer  time  delays.  There  was  no  other  traffic  on  the  network,  the  cable  access 
delay  was  negligible,  the  packet  transmission  time  at  10  MBS  was  only  67 .2 
microseconds  for  a  minimum  size  ETHERNET  packet,  and  the  propagation  delay  is 
negligible.  Therefore,  the  59  milliseconds  must  be  attributed  to  the  virtual 
circuit  protocol  operations  in  the  NIUs.  How  this  time  is  divided  between 
SEND  and  RECEIVE  functions  is  not  yet  known.  (We  hope  to  evaluate  that 
later.) 

The  throughput  and  delivery  delays  achieveable  with  LAN  systems  of  the 
type  discussed  here  without  loading  effects  depend  primarily  on  the  packet 
building  and  buffer  management  processes.  The  media  transfer  speeds,  access 
management  and  other  activities  are  much  less  significant.  We  feel  that  there 
are  three  classes  of  loading  that  may  have  significant  effects  on  throughput 
and  delivery  delay,  and  we  are  now  ready  to  investigate  these  more  fully. 
Class  One  is  the  activity  at  the  input  port  and  the  contention  for  processing 
cycles  within  a  single  Port  Interface  Controller  of  interest.  The  Second 
Class  is  the  total  combined  traffic  at  all  of  the  ports  on  a  Cable  Interface 
Unit,  or  the  multiplexing  that  is  common  in  many  LANs.  The  Third  Class  is  the 
total  oombined  traffic  on  the  media  that  causes  collisions,  retrys,  and  delays 
in  emptying  the  transmit  buffer.  These  are  the  delays  that  prevent  the  CID 
from  disposing  of  the  packets  and  receiving  the  required  acknowledgements.  We 
have  presented  a  sample  of  Class  One  and  we  have  studied  Class  Two.  The 
ability  to  study  Three  requires  not  only  an  initimate  understanding  of  Classes 
One  and  Two,  and  the  capability  to  measure  the  parameters  associated  with 


them,  but  also  the  capability  to  precisely  control  the  media  loading.  Ve 
believe  that  One  and  Two  are  similar  in  magnitude  and  effect  and  our  results 
indicate  they  are  comparable.  He  have  not  been  able  to  study  Class  Three  as 
of  yet  but  we  are  moving  in  that  direction.  He  have  observed  buffer  starva¬ 
tion  and  the  invocation  of  flow  control  under  only  Class  One.  It  is  obvious 
that  the  inability  to  empty  the  TRANSMIT-BUFFER  will  exergerate  the  buffer 
starvation  problem.  He  do  not  know  the  magnitude  of  this  effect  at  this  time. 

A  full  understanding  of  systems  such  as  these  and  the  capability  to 
predict  their  performance  depends  almost  entirely  on  the  ability  to  develope  a 
complete  timing  model  such  as  the  one  shown  in  Figure  6  and  the  ability  to 
quantify  the  various  time  intervals  involved.  It  should  be  noted  that 
although  "queues”  are  utilized,  a  queuing  theory  analysis  is  not  totally 
sufficient  due  to  the  real  life  effects  of  the  interference  and  interaction 
between  the  different  processes  that  cannot  be  handled  in  classical  queueing 
theory. 

For  example,  consider  the  Internal  Transfer  from  the  SEND-QUEUE  to  the 
TRANSMIT-BUFFER  as  shown  in  Figure  6.  The  packet  to  be  copied  or  transferred 
is  located  in  the  same  memory  that  is  used  to  accumulate  the  characters  from 
the  ports  and  assemble  the  packets.  Hhen  a  Port  Interface  Chip  has  received  a 
oomplete  character  it  must  be  given  memory  access  to  transfer  that  character 
Into  the  PORT-BUFFER.  These  memory  cycles  given  to  the  Ports  are  then  not 
available  for  the  Internal  Transfer  to  the  TRANSMIT-BUFFER.  The  result  is 
that  Increases  in  the  Interface  Controller  load  from  higher  character  input 
rates,  cause  delays  in  the  movement  of  complete  packets  from  the  Interface 
Controller  reducing  the  overall  transfer  rate  just  when  it  needs  to  increase  I 

Our  analysis  shows  that,  with  the  exception  of  the  transfers  to  and  from 
the  Ports,  all  time  intervals  are  random  variables  whose  properties  are 
difficult  to  predict  and  must  be  studied  during  actual  operations.  He  are  at 


a  point  now  to  begin  these  studies.  We  have  identified  the  particular 
measurements  needed,  and  we  have  a  better  idea  of  what  test  tools  are  needed 


to  measure  them. 
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